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Abstract 


Large,  complex  systems  development  has  always  been  challenging,  even  when  the 
“only”  things  a  program  manager  had  to  worry  about  were  cost,  schedule,  and  per¬ 
formance  within  a  single  program.  The  emergence  of  operational  concepts  such  as 
network-centric  operations,  greatly  expanded  use  of  joint  and  combined  operations, 
and  rampant  growth  in  system  complexity  has  led  to  the  prevalence  of  interoper¬ 
able  systems  of  systems  as  the  preferred  solution  to  providing  operational  capabil¬ 
ity.  This  report  explores  how  systems-of-systems  realities  necessitate  changes  in 
the  processes  used  to  acquire,  develop,  field,  and  sustain  operational  capability. 
Interoperable  acquisition  is  defined,  and  key  concepts  are  explored  through  an 
analysis  of  some  of  the  ways  in  which  traditional  (i.e.,  system-centric)  acquisition 
approaches  can  result  in  problems  when  applied  to  a  system-of-systems  context. 
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1  Introduction 


Before  discussing  the  challenges  in  practicing  interoperable  acquisition,  it  is  neces¬ 
sary  to  define  the  term  and  explain  its  significance.  As  used  in  this  report,  interop¬ 
erable  acquisition  consists  of  the  set  of  practices  that  enable  acquisition,  develop¬ 
ment,  and  operational  organizations  to  collaborate  more  effectively  to  field 
interoperable  systems.  These  practices  are  distinguishable  from  the  processes  used 
to  acquire  individual  systems,  in  that  the  purpose  of  interoperable  acquisition  is  to 
influence  the  acquisition  behavior  of  the  constituents1  to  maximize  the  likelihood 
of  a  successful  system-of-systems  outcome,  as  opposed  to  maximizing  the  out¬ 
come  for  any  individual  system. 

Why  is  this  distinction  important?  Traditional  system  acquisition — as  practiced 
“down  through  the  ages” — focuses  on  achieving  specified  performance  objectives 
(including  functional  requirements,  like  such-and-such  throughput  and  so-many 
transactions  per  second,  as  well  as  nonfunctional  requirements  such  as  maintain¬ 
ability  and  reliability)  within  cost  and  schedule  constraints.  Each  system  is  devel¬ 
oped  in  response  to  a  perceived  operational  mission  need  and  has  its  own  opera¬ 
tional  requirements,  community  of  interest,  funding,  and  the  like.  This  system¬ 
centric  approach  has  resulted  in  the  proliferation  of  stovepiped  systems  that  are 
integrable  with  difficulty — -if  at  all. 

However,  system-of-systems  thinking,  as  epitomized  by  the  network-centric  war¬ 
fare  concept,  demands  a  degree  of  collaboration  and  coordination  among  constitu¬ 
ent  systems  that  cannot  be  achieved  by  “adding  in”  something  to  stovepiped  sys¬ 
tems  (any  more  than  maintainability,  reliability,  or  any  other  desirable  quality  can 
be  added  in).  Instead,  the  acquisition  process — from  initial  recognition  of  the  need 
for  a  material  solution  to  the  eventual  retirement  of  a  system — must  be  informed  by 
the  system-of-systems  realities  and  account  for  their  influences.  Doing  so  requires 
that  the  practitioner  possess  an  understanding  of  the 

•  characteristics  of  a  system  of  systems 

•  aspects  of  interoperability  and  the  ways  in  which  they  affect  the  acquisition, 
development,  deployment,  sustainment,  and  operational  use  of  the  system  of 
systems 

Imparting  that  understanding  is  the  focus  of  this  report.2 


1  The  term  constituents  encompasses  all  automated,  mechanized,  or  human  elements — including  pro¬ 
gram  offices,  contractors,  and  systems — that  have  a  role  in  a  system  of  systems. 

2  This  report  is  one  of  four  technical  notes  resulting  from  a  Carnegie  Mellon®  Software  Engineering  Insti¬ 
tute  (SEI)  independent  research  and  development  (IR&D)  effort  entitled  Toward  Interoperable  Acquisi¬ 
tion.  The  other  technical  notes  examine  risk  management,  schedule,  and  process  considerations  for  in¬ 
teroperable  acquisition.  A  fifth  technical  note,  partially  supported  by  the  same  IR&D  effort,  will  deal  with 
programmatic  interoperability  issues.  (Carnegie  Mellon  is  registered  in  the  U.  S.  Patent  and  Trademark 
Office  by  Carnegie  Mellon  University.) 
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The  rest  of  this  report  is  organized  in  the  following  manner: 

•  Section  2  presents  several  examples  that  illustrate  how  the  changing  nature  of 
systems — and  systems  of  systems — and  the  evolution  in  acquisition  philoso¬ 
phies  have  resulted  in  significant  obstacles  for  the  U.S.  Department  of  De¬ 
fense  (DoD)  and  other  Executive  Branch  agencies. 

•  Section  3  provides  a  detailed  discussion  of  the  challenges  of  interoperable 
acquisition  and  describes  how  specific  issues  manifest  themselves. 

•  Section  4  contains  a  brief  summary  of  this  report  and  suggests  potential  future 
efforts. 
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2  Background 


As  we  began  our  analysis  of  the  interoperable  acquisition  “problem  space,”  we 
sought  to  contrast  our  experiences  with  interoperability  with  the  challenges  arising 
from  the  increasing  demands  for  interoperability  in  a  network-centric  world.  What 
distinguishes  today’s  challenges  is  the  complexity  of  current — and  projected — 
systems  and  the  resulting  convoluted  cross-system  dependencies.  This  complexity 
has  greatly  exacerbated  the  acquisition  difficulties  confronting  the  DoD. 

In  the  DoD  acquisition  environment  of  the  1980s,  most  systems  were  the  responsi¬ 
bility  of  a  single  prime  contractor.  Integrating  acquisition  efforts  across  systems  fell 
to  the  government.  The  B-1B  program,  with  four  contractors  performing  as  primes 
and  the  government  acting  as  the  integrator,  exemplified  this  approach — and  ex¬ 
posed  its  shortcomings.  From  that  program,  it  became  apparent  that  successful  de¬ 
velopment  of  increasingly  complex  systems  required  that  the  government  allow  its 
prime  contractor  more  latitude  in  selecting  components  and  integrating  them  into 
the  final  system.  Thus,  in  strong  contrast  to  the  approach  taken  in  the  B-1B  pro¬ 
gram,  Northrop  Grumman  was  given  broad  latitude  to  decide  what  components 
became  part  of  the  B-2  system  of  systems  and  how  to  integrate  them.  The  approach 
taken  in  the  B-2  program  was  widely  viewed  as  more  successful.  Similar  ap¬ 
proaches  have  been  successfully  employed  across  all  of  the  services,  from  the 
Army’s  Abrams  main  battle  tank  to  the  Navy’s  Submarine  Launched  Ballistic  Mis¬ 
sile  program. 

However,  sometimes  challenges  came  in  smaller  “boxes”:  Programs  like  the  Joint 
Tactical  Information  Distribution  System  (JTIDS)  and  its  successor,  the  Enhanced 
JTIDS  System  (EJS),  had  many  interoperability  challenges.  The  multiservice,  mul¬ 
tiplatform  aspects  of  those  programs  challenged  the  acquisition  system  and  the 
various  program  offices  needing  to  use  or  interface  with  JTIDS  and  EJS. 

Upgrading  existing  platforms  has  also  provided  challenges  over  the  years,  making 
configuration  management  one  of  the  classic  difficulties.  For  example,  in  updating 
the  aging  C-130  Hercules  fleet,  the  Air  Force  has  seen  that  few  examples  of  the  C- 
13  OH,  the  main  tactical  transport  in  the  DoD  inventory,  are  in  fact  the  “same”  as 
the  airplane  created  just  before  or  just  after  it.  In  some  cases,  these  differences  are 
minor;  in  others,  they  are  significant. 

In  addition,  acquisition  guidance  provided  to  program  offices,  most  often  in  the 
form  of  a  Program  Management  Directive  (PMD),  has  been  very  program- specific. 
The  PMD  provided  broad  guidance  to  the  program  office  on  what  activities  needed 
to  be  addressed  with  the  yearly  funding  provided.  A  PMD  typically  addressed  in¬ 
teroperability  only  in  vague  terms  for  point-to-point  requirements — if  at  all.  Any 
notion  of  a  system  of  systems  with  ubiquitous  interoperability  available  “on  the 
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fly” — a  primary  tenet  of  network-centric  warfare — was  nonexistent  (as  were  its 
operational  requirements  and  the  funding  to  provide  it). 

Over  the  years  since  the  1980s,  advances  in  control  and  communications  capabili¬ 
ties  have  had  a  powerful  impact  on  the  DoD’s  need  for  interoperability.  This  im¬ 
pact  is  further  intensified  by  a  new  web  of  alliances,  very  different  from  the  1980s 
where  there  were  relatively  clear  lines  of  demarcation  between  “blue”  forces  and 
their  “red”  adversaries.  Today,  a  new  ally  may  have  systems  that  were  totally  un¬ 
known  to  the  DoD  a  few  years  before  or  were  part  of  an  adversarial  network  that 
was  a  threat. 

On  current  systems  in  development,  we  see  another  phenomenon  that  adds  to  the 
concern:  the  absence  of  overall  responsibility  for  the  system  of  systems  acquisition. 
This  deficiency  can  be  illustrated  through  an  example  from  the  communications 
domain.  The  ongoing  modernization  of  the  extremely  high  frequency  (EHF)  mili¬ 
tary  satellite  communications  (MILSATCOM)  systems  is  the  focus  of  a  number  of 
programs  spanning  all  branches  of  the  military  (i.e.,  Army,  Navy,  and  Air  Force) 
and  encompassing  several  distinct  organizations.  Within  the  Air  Force,  there  is  the 
MILSATCOM  Systems  Wing  (MCSW)  at  the  Space  and  Missile  Systems  Center 
(SMC),  which  is  under  the  Air  Force  Space  Command;  the  Air  Force  terminal  pro¬ 
gram  office  is  part  of  the  Electronic  Systems  Center  (ESC),  which  is  a  component 
the  Air  Force  Material  Command.  In  addition,  the  Air  Force  terminal  program  of¬ 
fice  is  “dual-hatted”  to  the  MCSW.  The  Army  and  Navy  each  have  their  own  com¬ 
munications  terminal  program  offices,  as  well.  To  provide  the  desired  EHF 
MILSATCOM  capabilities  to  the  warfighter,  several  new  systems  have  to  be  de¬ 
ployed — including  the  satellite,  satellite  command  and  control  system,  communica¬ 
tions  terminals,  and  mission  control  system.  Naturally,  the  satellite  development 
received  most  of  the  management  attention  (since  it  is  the  single  most  expensive 
component — on  a  per-unit  basis — of  the  MILSATCOM  system  of  systems).  How¬ 
ever,  there  was  no  overall  responsibility  for  ensuring  that  all  of  the  parts  of  the  sys¬ 
tem  of  systems  were  considered  together.  An  early  consequence  of  this  became 
apparent  when  funding  instability  caused  a  critical  terminal  component  to  be  de¬ 
layed  to  the  point  that  it  missed  an  important  installation  “window”  for  a  high- 
priority  operational  platform.  Because  of  the  operational  and  maintenance  sched¬ 
ules  for  this  platform,  the  next  opportunity  to  install  this  component  will  be  five 
years  later,  adversely  affecting  the  schedule  for  the  overall  EHF  MILSATCOM 
modernization. 

We  believe  that  such  consequences  can  be  acknowledged,  addressed,  and  often 
mitigated  by  applying  risk  management  principles.  The  difference  from  the  past 
approach  is  to  move  the  perspective  up  from  the  risks  to  a  particular  system  to  the 
more  inclusive  system  of  systems.  Some  risks  are — in  fact — best  addressed  at  the 
level  of  a  particular  system.  Most  system-of-system  risks,  however,  benefit  from  a 
shared  perspective  across  two  or  more  of  the  systems  responsible  for  providing  the 
desired  mission  capability.  In  systems  of  systems,  in  general,  all  will  benefit  from 
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shared  information  about  the  risks  that  each  is  facing  in  an  unstable,  dynamic  world 
of  changing  funding  and  requirements. 
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3  Discussion 


Changing  perspective  from  a  single  system  interoperating  in  some  federated  fash¬ 
ion  with  other  systems  to  a  system  of  systems  alters  the  way  in  which  risks  must  be 
considered  and  mitigated.  A  system-centric  approach  will — in  general — fail  to  ob¬ 
tain  the  best  possible  outcome  for  the  system  of  systems.  To  understand  the  reasons 
for  this,  it  is  necessary  to 

•  understand  some  critical  aspects  of  systems  of  systems  and  interoperability 

•  determine  how  those  aspects  affect  the  acquisition,  development,  deployment, 
sustainment,  and  operational  use  of  the  system  of  systems 

Traditionally,  interoperability  has  been  considered  an  operational  characteristic  of 
an  aggregation  of  systems.  In  other  words,  systems  were  interoperable  when  they 
could  exchange  information  successfully. 

In  a  previous  SEI  IR&D  effort,  Morris  and  team  determined  that  considering  only 
the  operational  aspect  of  interoperability  was  insufficient  in  an  acquisition  context. 
That  team  defined  three  aspects  that  must  be  considered  jointly:  (1)  programmatic, 
(2)  constructive,  and  (3)  operational  [Morris  04].  One  consequence  of  this  model  is 
the  insight  that  focusing  on  only  one  aspect  will  not  ensure  interoperability — any 
more  than  merely  thinking  about  only  one  piece  of  a  system  of  systems  will 
achieve  it. 

The  following  sections  will  delve  into  system-of-systems  dynamics  and  how  the 
programmatic,  constructive,  and  operational  aspects  of  interoperability  affect  vari¬ 
ous  acquisition  processes — including  risk  management. 

3.1  SYSTEM  CENTRIC  VERSUS  SYSTEM  OF  SYSTEMS 

A  system  of  systems  has  characteristics  that  are  fundamentally  different  from  those 
of  a  single  system.  These  differences  are  summarized  in  Table  1. 
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Table  1:  Single  Systems  Versus  Systems  of  Systems3 


Issue 

Single  System 

System  of  Systems 

Changing  and  potentially  unknown  constitu¬ 
ents 

Constituents 

All  constituents  are 
known  and  visible 

Entity  assembling  a  system  of  systems  may 
not  know  constituents  until  runtime. 

Constituent  may  not  know  it  is  part  of  a  system 
of  systems. 

Purpose 

Predetermined  by 
system  owner  and 
conveyed  to  con¬ 
stituents 

Continuously  evolving,  cooperatively  deter¬ 
mined,  and  may  or  may  not  be  known  by  sys¬ 
tems  participating  in  systems  of  systems 

Control 

Hierarchically  struc¬ 
tured  with  central 
control  by  system 
owner 

System  owners  participating  in  a  system  of 
systems  may  have  control  over  their  systems, 
but  they  do  not  control  how  and  when  their 
systems  are  used  in  the  system  of  systems. 

Entity  assembling  a  system  of  systems  has 
control  over  assembly  but  not  over  the  partici¬ 
pating  systems. 

Requirements 

Defined  and  man¬ 
aged  by  system 
owner 

Systems  participating  in  the  system  of  systems 
often  have  to  anticipate  how  their  system  will 
be  used. 

Ownership 

Pieces  developed 
are  owned,  main¬ 
tained,  and  evolved 
by  system  owner. 

Constituent  systems  are  independently  owned, 
developed,  maintained,  and  evolved. 

Boundaries 

Closed  with  clearly 
defined  boundaries 

In  general,  unbounded  and  part  of  larger  sys¬ 
tems  of  systems 

Visibility 

All  aspects  can  be 
seen,  understood, 
and  controlled. 

Components  and  process  aspects  are  beyond 
control  and  visibility  of  developers,  users,  and 
owners. 

In  addition,  systems  of  systems  exhibit  emergent  behavior.  In  other  words,  a  system 
of  systems  .  .performs  functions  and  carries  out  purposes  that  do  not  reside  in  any 
component  system”  [Maier  96,  discussed  in  Fisher  06].  While  emergence  can  result 
in  negative  outcomes  (e.g.,  the  electrical  power  grid  failure  of  2004  in  the  north¬ 
eastern  United  States),  there  would  be  no  systems  of  systems  if  not  for  it.  The  goal 

3  The  contents  of  this  table  are  taken  from  “Will  My  System  ‘Play  Nicely’  with  Others?  A  Tutorial  Explor¬ 
ing  CMMI  to  Improve  Systems  of  Systems  Success”  presented  at  the  Software  Engineering  Process 
Group  conference  in  2006  (SEPG  2006)  by  Lisa  Brownsword,  Suzanne  Garcia,  and  Grace  Lewis  of  the 
SEI. 


SOFTWARE  ENGINEERING  INSTITUTE  |  7 


for  the  system  of  systems  is  realized  only  through  the  emergent  behavior  of  the 
constituent  components. 

3.2  THREE  ASPECTS  OF  INTEROPERABILITY 

Interoperability  has  traditionally  been  defined  in  an  operational  context  (e.g.,  the 
ability  of  systems  to  exchange  information).  This  definition  is  too  imprecise  and 
incomplete  to  describe  the  essential  characteristics  of  interoperability,  much  less  to 
allow  one  to  reason  about  possible  strategies  to  achieve — and  maintain — 
interoperability.  In  System  of  Systems  Interoperability  (SOSI):  Final  Report,  Morris 
and  associates  discuss  how  interoperability  is  not  a  property  of  a  system  in  isola¬ 
tion,  but  is  dependent  on  a  particular  context  [Morris  04].  Specifically,  they  define 
interoperability  as 

the  ability  of  a  set  of  communicating  entities  to  (1)  exchange  specified  state 
data  and  (2)  operate  on  that  state  data  according  to  specified,  agreed- 
upon,  operational  semantics 

While  this  definition  addresses  the  issue  of  context,  it  doesn’t  go  far  enough.  The 
SOSI  report  further  identifies  three  distinct — but  interrelated — aspects  that,  taken 
together,  provide  a  richer  understanding  of  what  is  meant  by  interoperability. 

Figure  1,  the  SOSI  model,  illustrates  the  programmatic,  constructive,  and  opera¬ 
tional  aspects  of  interoperability  and  their  relationships  within  and  across  programs 
[Morris  04,  Meyers  05]. 


Program-1  Program-! 


Figure  1:  The  SOSI  Model 

The  three  interoperability  aspects  are  characterized  as  follows: 

•  Operational  interoperability  is  closely  aligned  with  the  traditional  definition  of 
interoperability  (the  ability  of  systems  to  exchange  relevant  information),  but 
it  adds  the  notion  of  compatible  (or  complementary)  operational  concepts. 
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•  Constructive  interoperability  reflects  the  degree  to  which  the  different  system 
design,  engineering,  and  production  processes  and  tools  are  able  to  exchange 
information  in  an  understandable  manner. 

•  Programmatic  interoperability  expresses  the  ability  of  programs  to  exchange 
meaningful  information  about  the  management  of  the  programs  involved.  This 
information  can  run  the  gamut  from  budget  and  schedule  information  to  de¬ 
tails  on  how  to  interpret  risks. 

The  emphasis  on  aspects  is  critical:  there  is  no  such  thing  as  a  programmatic,  con¬ 
structive,  or  operational  interoperability  issue  per  se.  Instead,  there  are  interopera¬ 
bility  issues  that  have  implications  or  “manifestations”  in  any  or  (in  most  cases)  all 
of  the  interoperability  aspects.  Whereas  traditional  treatments  of  interoperability 
largely  ignore  the  constructive  and  programmatic  aspects,  the  participants  in  the 
SO  SI  study  concluded  that  the  impact  of  those  aspects  on  interoperability  is  signifi¬ 
cant.  In  fact,  they  concluded  that  the  programmatic  interoperability  aspects  often 
overwhelm  the  operational  and  constructive  aspects. 

3.3  IMPLICATIONS  FOR  ACQUISITION 

The  confluence  of  system-of-system  characteristics  and  interoperability  aspects 
frequently  limits  success  when  traditional  systems  acquisition  processes  are  ap¬ 
plied.  The  paradigm  embodied  by  the  conventional  approaches  to  solving  the  ac¬ 
quisition  problem — thinking  of  a  system  of  systems  as  a  collection  of  individual 
systems  conceived,  managed,  developed,  and  fielded  individually — doesn’t  account 
for 

•  the  interdependencies  between  components  of  a  system  of  systems 

•  the  ways  in  which  individual  system  engineering  or  acquisition  decisions  must 
influence  (and,  in  turn,  be  influenced  by)  the  broader  needs  of  the  system  of 
systems 

While  the  limitations  of  traditional  approaches  are  widely  acknowledged,  there  has 
not  been  widespread  recognition  that  something  new  needs  to  be  done.  Indeed,  the 
conventional  wisdom  appears  to  be  that  simply  performing  the  traditional  activi¬ 
ties — but  somehow  doing  them  better  or  more  frequently — will  rectify  this  appar¬ 
ent  inconsistency.  Thomas  Kuhn,  in  his  seminal  work  The  Structure  of  Scientific 
Revolutions ,  described  this  approach  as  “normal  science.”  Kuhn’s  point  is  that  peo¬ 
ple  resist  changing  their  thinking  even  when  it  is  obvious  that  the  conventional 
wisdom  increasingly  fails  to  account  for  observed  behavior  [Kuhn  62].  In  fact,  until 
there  is  a  failure  that  is  sufficiently  painful  to  force  a  willingness  to  entertain  new 
approaches,  the  old  paradigm  will  persist. 

However,  you  cannot  stop  performing  any  of  the  traditional  systems  engineering  or 
acquisition  activities.  A  parallel  can  be  drawn  with  the  influx  of  commercial  off- 
the-shelf  (COTS)  products  into  DoD  systems  a  decade  ago:  many  acquisition  or¬ 
ganizations  stopped  performing  critical  systems  engineering,  acquisition,  and  test¬ 
ing  activities  because  “the  COTS  vendors  already  did  that.”  The  ensuing  failures 


SOFTWARE  ENGINEERING  INSTITUTE  |  9 


amply  demonstrated  the  fallacy  of  this  belief  The  DoD  is  still  coming  to  grips  with 
the  realization  that  all  of  the  traditional  activities  must  be  carried  out  with  COTS 
products,  but  in  ways  that  are  subtly — and  significantly — altered.  For  example, 
systems  engineers  must  focus  less  on  exquisitely  refining  system  performance  re¬ 
quirements  and  spend  much  more  time  understanding  how  the  interplay  of  their 
requirements,  and  a  suite  of  COTS  products,  affects  the  system  performance.  This 
shift  in  emphasis  requires  the  development  of  several  new  capabilities,  including 

•  reverse-engineering  the  interactions  between  COTS  products  (through  a  mix¬ 
ture  of  “white  box,”  “gray  box,”  and  “black  box”  analysis  techniques) 

•  “getting  inside  the  mind”  of  the  user  community — and  eliciting  their  help — in 
understanding  what  is  really  a  requirement  and  where  there  is  flexibility 

Similarly,  new  skills  are  required  in  every  other  discipline  involved  in  system  ac¬ 
quisition — including  testing,  sustainment,  and  training. 

To  help  make  these  implications  more  concrete,  we  will  present  two  vignettes — 
based  on  actual  acquisition  programs — and  two  summaries  of  workshops  that  illus¬ 
trate  some  of  the  types  of  problems  that  can  arise  from  treating  a  system  of  systems 
as  simply  a  collection  of  individual  systems. 

3.3.1  Failure  of  Program-Centric  Risk  Management 

In  this  vignette,  there  is  a  large,  complex  system  of  systems  with  largely  independ¬ 
ent  program  offices — under  the  same  directorate — responsible  for  managing  all 
aspects  of  the  procurement,  development,  and  deployment  of  their  respective  com¬ 
ponents  of  the  system  of  systems.  There  are  numerous  interrelationships  between 
the  various  offices  and  with  other  organizations  not  part  of  the  directorate,  includ¬ 
ing  requirements  for  command  and  control  interoperability  and  schedule  dependen¬ 
cies.  Figure  2  depicts  a  selected  subset  of  these  relationships. 
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Figure  2:  System-of-Sy stems  Context  Interoperability  Map4 

Each  program  office  depicted  in  the  figure  has  its  own  risk  management  program, 
complete  with  independent  risk  reduction  efforts.  Program  Office  “A”  had  been 
pursuing  a  risk  reduction  effort  with  Contractor  “Z”  that  was  intended  to  mitigate 
any  risks  to  their  system  (built  by  Contractor  “X”)  arising  from  delays  on  the  part 
of  Program  Office  “B”  and  Contractor  “Y.”  Based  on  quarterly  program  reviews 
conducted  at  the  directorate  level,  “A”  concluded  that  it  could  safely  discontinue  its 
risk  reduction  effort  as  “B”  was  projected  to  deliver  its  capability  well  in  advance 
of  “X’s”  deadline,  as  shown  in  Figure  3.  Cancelling  this  risk-reduction  effort  would 
save  in  excess  of  $1  million. 
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Figure  3:  “ Official  ”  Integrated  Schedules 


Unfortunately,  contractual  technical  requirements  synchronization  issues  between 
“B”  and  “Y”  had  reached  the  point  where  it  would  take  more  than  one  year  to 


4  The  concept  of  interoperability  maps  is  discussed  in  a  technical  note  that  introduces  the  System-of- 
Systems  Navigator,  an  integrated  set  of  practices  that  address  the  challenges  related  to  achieving  sys- 
tem-of-systems  interoperability  [Brownsword  06]. 
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“catch  up”  to  all  of  the  baseline  changes  incorporated  by  “X.”  In  addition,  there 
was  a  problem  with  the  delivery  schedule  for  the  product  supplied  by  Contractor 
“P”  (via  Agency  “Q”)  to  “Y.”  The  net  effect  of  these  delays  was  that  instead  of 
slack  in  the  schedule  dependency  between  “A”  and  “B,”  there  was  a  significant  gap 
between  the  system-of-systems  deadline  and  “B’s”  ability  to  deliver,  resulting  in 
“B”  failing  to  provide  a  necessary  command  and  control  interoperability  capability 
on  time.  The  gap  is  shown  in  Figure  4. 


Integrated  Program  Schedule 


Today 


Avhidt  Negative 
Schedule  Sleek 


Figure  4:  “ Actual  ”  Integrated  Schedules 


When  these  issues  were  finally  understood,  it  became  apparent  that  the  only  way  to 
preserve  “A’s”  schedule — and  thus  avoid  a  major  program  breach — was  to  restart 
the  previously  cancelled  risk  reduction  effort.  The  startup  costs  and  compressed 
schedule  resulted  in  an  estimated  cost  to  complete  the  restarted  risk  reduction  effort 
that  was  several  times  greater  than  the  originally  hoped-for  savings. 


This  simple  example  illustrates  the  failure  brought  on  by  approaching  risk  man¬ 
agement  in  a  program-centric  fashion  for  a  system  of  systems.  Had  the  programs, 
instead,  pursued  a  risk  management  approach  for  the  system  of  systems,  the  linkage 
between  the  “A’s  ”  risk  reduction  effort  (with  “Z  ”)  and  the  risks  and  problems  pre¬ 
sent  in  “B’s  ”  program  would  have  been  immediately  apparent.  In  that  event,  it  is 
unlikely  that  the  decision  to  cancel  “A’s  ”  risk  reduction  effort  would  have  been 
made.  Optimizing  the  risk  management  from  the  perspective  of  any  individual  pro¬ 
gram  will — in  general — result  in  suboptimal  risk  management  across  a  system  of 
systems.  In  fact,  risk  identification  and  mitigation  strategies  that  are  well  suited  to 
individual  programs  may  fail  spectacularly  when  applied  in  a  system-of-systems 
context.  This  observation  is  true  because  many  of  the  risks  in  the  system  of  systems 
arise  from  emergent  behaviors  that  result  from  the  interrelationships  between  the 
component  systems  and  organizations  and — in  fact — reside  in  no  one  individual 
program. 


3.3.2  Absence  of  System-of-Systems  Engineering 

At  a  recent  workshop  on  requirements  management  in  a  system-of-systems  context, 
participants  described  some  of  the  challenges  arising  from  the  disconnect  between 
(a)  the  goals  for  creating  a  system  of  systems  and  (b)  the  organizational  interrela¬ 
tionships  between  the  various  entities  responsible  for  acquiring,  developing,  field- 
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ing,  and  sustaining  the  individual  constituents  that  compose  the  system  of  systems 
[Meyers  06]. 

In  particular,  participants  noted  that  the  absence  of  any  organizational  entity  with 
sufficient  authority  and  responsibility  greatly  complicates  the  accomplishment  of 
tradeoff  analyses  across  the  systems  in  a  system  of  systems.5  These  relationships — 
as  described  by  the  workshop  participants — can  be  represented  by  patterns  and 
antipatterns.  Gamma  and  associates  defined  software  patterns  as  standard  solu¬ 
tions  to  common  software  engineering  problems  [Gamma  94].  Andrew  Koenig  de¬ 
fined  antipatterns  as  either  known  bad  solutions  to  common  problems  or  as  a  way 
to  proceed  from  a  bad  situation  to  a  better  one  [Koenig  95].  Brown,  McCormick, 
and  Thomas  have  similarly  defined  patterns  and  antipattems  for  project  manage¬ 
ment  [Brown  00].  For  our  purposes,  we  have  extended  the  definition  of  antipattems 
to  encompass  the  absence  of  appropriate  patterns.  In  this  context,  Figure  5  repre¬ 
sents  the  patterns  and  antipattems  identified  by  workshop  participants  that  are  re¬ 
lated  to  requirements  management  in  a  system  within  a  system  of  systems.  (Note: 
antipattems  are  represented  with  underscores  to  denote  the  negation  of  the  actor[s] 
or  their  relation,  as  appropriate.) 


89fi 


System  Engineering 


•iA)  optimal 


multiple 

organizations 


Figure  5:  System-of-Systems  Requirements  Tradeoffs,  Patterns,  and  Antipatterns 


5  Figure  5  and  its  discussion  are  taken  from  an  SEI  technical  note  entitled  Programmatic  Interoperability 
that  is  in  development. 
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The  absence  of  a  systems  engineering  capability  at  the  system-of-systems  level 
(shown  in  the  figure  as  the  SoS  System  Engineering  Organization  antipattem), 
results  in  the  inability  to  adjudicate  conflicting  demands  whose  scope  extends  be¬ 
yond  that  of  any  single  system. 

Out  of  many  possible  classes  of  failure  that  arise  from  treating  a  system  of  systems 
as  simply  a  collection  of  individual  systems,  Figure  5  illustrates  this  one:  There  are 
some  new  activities — like  performing  cross-system  tradeoffs — that  cannot  easily  be 
accommodated  within  a  program-centric  organizational  structure.  While  the  par¬ 
ticipants  in  this  workshop  focused  on  requirements  management,  this  same  class  of 
failure  exists  for  any  aspect  of  a  system-of-systems  acquisition,  including  funding 
and  schedule  tradeoffs. 

3.3.3  Disconnect  Between  System-of-Systems  Operations  and 
Acquisition 

During  the  course  of  a  workshop  on  acquisition  issues  affecting  the  Army’s  trans¬ 
formation  to  a  “Future  Force,”  it  was  obvious  that  success  in  a  system-of-systems 
environment  requires  an  understanding  of  the  relationships  between  operational 
interoperability  decisions  governing  how  forces  are  to  be  deployed  (e.g.,  opera¬ 
tional  environment,  operational/employment  concepts,  and  interoperability  rela¬ 
tionships)  and  the  corresponding  programmatic  interoperability  strategy  for  acquir¬ 
ing,  fielding,  and  sustaining  the  systems  to  be  used  by  those  forces  [Smith  05].  In 
other  words,  there  needs  to  be  a  linkage  between  the  operational  and  programmatic 
interoperability  aspects  of  the  system  of  systems.  This  cross-aspect  linkage  is 
shown  in  Figure  6  as  the  heavy  double-ended  arrow  between  force _structure  and 
acq_stratQgy. 

One  of  the  key  findings  from  this  workshop  was  that — in  today’s  acquisition  envi¬ 
ronment — there  are  significant  disconnects  between  how  systems  are  fielded  and 
operational  forces  are  deployed.  Specifically,  the  linkage  between  the  system-of- 
systems  operational  concepts  and  force  structure — and  the  corresponding  acquisi¬ 
tion  strategies — is  inadequate  to  ensure  harmonization  between  the  acquisition 
processes  and  the  operational  force.  Another  example  of  this  disconnect  is  found  in 
the  disparity  between  how  individual  program  offices  conduct  their  analysis  of  al¬ 
ternatives  and  how  capability  is  assessed  in  an  operational  environment. 

As  a  consequence,  interoperability  (and  therefore  mission  effectiveness  and  opera¬ 
tional  capability)  often  suffers  during  the  transition  from  old  systems  to  new  ones, 
as  individual  systems  are  deployed  to  Army  units. 
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into 


Figure  6:  Linkage  Between  System  Acquisition  and  Force  Capability 
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:  Cross-aspect  relationship 


The  problem  exposed  by  the  workshop  can  easily  be  illustrated  with  a  simple  sce¬ 
nario:  Suppose  there  are  two  units,  “X”  and  “Y,”  that  are  deployed  to  the  same  op¬ 
erational  theater  and  are  required  to  coordinate/collaborate  with  each  other  in  carry¬ 
ing  out  their  assigned  mission.  The  units  are  equipped  with  systems  SA  and  SB, 
respectively. 

•  For  X  and  Y  to  perform  this  coordination/collaboration,  SA  and  SB  are  re¬ 
quired  to  be — and  are  currently — interoperable.  This  state  is  shown  in  Figure 
7. 


SA  and  SB  are  acquired,  developed,  and  fielded  by  separate  program  offices 
(PMOa  and  PMOB)  in  response  to  separate  operational  requirements,  with 
separate  funding  sources,  modernization  plans,  and  the  like.  The  interoperabil¬ 
ity  between  the  two  systems  is  characterized  by  conformance  to  a  standard 
protocol,  P,  which  consists  of  several  “pieces”  (Pi,  P2,  and  P3),  as  shown  in 
Figure  8. 
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•  In  the  course  of  each  program’s  independently  planned  upgrades  (to  SA  and 
SB)  and  based  on  their  respective  prioritized  requirements,  risk  profiles,  fund¬ 
ing  availabilities,  and  other  factors,  the  two  program  offices  decide  to  imple¬ 
ment  the  upgrades  incrementally.  Operating  independently,  PMOA  decides  to 
implement  upgrades  to  protocol  P  in  the  order  Pi,  P2,  and  P3.  However, 

PMOb — for  equally  valid  reasons — decides  to  implement  upgrades  to  P  in  a 
different  order:  P3,  Pi,  and  P2.  Throughout  the  implementation,  there  are  peri¬ 
odic  deployments  of  both  systems,  whatever  their  state  of  upgrade.  For  exam¬ 
ple,  there  might  be  a  deployment  of  SA  and  SB  with  just  the  Pi  portion  of  the 
SA  upgrade,  while  SB  has  P3  and  P2  incorporated.  That  circumstance  and  other 
disconnects  are  depicted  in  Figure  9. 
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Figure  9:  Unsynchronized  Upgrades  Resulting  in  Loss  of  Interoperability 

•  Asa  result  of  the  uncoordinated  upgrade  schedules  for  SA  and  SB  and  the  de¬ 
ployment  schedules  for  the  relevant  units,  the  previously  existing  interopera¬ 
bility  was  lost  and  wasn’t  regained  until  after  both  program  offices  completed 
the  incremental  upgrades  to  their  systems  (see  area  headed  “IV”  in  Figure  9). 
During  the  time  that  upgrades  were  not  synchronized,  there  was  a  loss  of  op¬ 
erational  capability  to  the  fielded  force. 

3.3.4  Limitations  of  a  Centralized  Management  Structure  in  a  System 
of  Systems 

In  this  vignette,  we  look  at  an  organization  responsible  for  the  acquisition  and  de¬ 
ployment  of  a  large,  complex  system  of  systems.  This  organization  was  structured 
as  a  relatively  loose  federation  of  peer-level  program  offices  under  the  auspices  of  a 
directorate,  as  shown  in  Figure  10.  Recently,  the  organization  had  experienced  a 
series  of  embarrassing  problems  that  only  became  apparent  at  the  1 1th  hour.  When 
later  examined,  these  problems  were  found  to  be  a  result  of  ineffective  communica¬ 
tions  among  various  system-of-systems  stakeholders  and  with  senior  leadership. 
Because  of  that  poor  communication,  decisions  were  made  on  incomplete  and,  of¬ 
ten,  incorrect  data. 
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Figure  10:  Federated  System-of -Systems  Organizational  Structure 

To  remedy  these  problems,  it  was  decided  to  centralize  all  programmatic  decisions: 
individual  programs  would  ensure  that  accurate  data  was  passed  “up  the  chain,” 
and  all  decisions  would  be  coordinated.  This  restructuring,  it  was  believed,  would 
end  the  problems  that  had  plagued  the  system  of  systems. 

The  resultant  organizational  structure  is  shown  in  Figure  1 1.  An  unforeseen  side 
effect  of  this  decision  was  that  the  time  needed  to  gather,  normalize,  and  aggregate 
the  required  data  and  to  prepare  the  necessary  documentation  to  support  the  deci¬ 
sion-making  process  was  significantly  longer  than  the  time  it  took  for  the  data  to 
become  outdated.  By  the  time  decisions  were  made,  the  circumstances  were  often 
so  changed  that  the  decision  was  no  longer  relevant. 


Figure  11:  Hierarchical  System-of-Systems  Organizational  Structure 
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The  circumstances  of  this  vignette  have  an  analogy  in  control  theory.  The  system  of 
systems  and  its  decision-making  process  can  be  considered  a  nonlinear  closed-loop 
feedback  control  system.6  The  control  loop  time  constant  is  the  time  required  to 
gather  and  interpret  the  data  and  make  a  decision;  the  system-of-systems  time  con¬ 
stant  is  the  rate  at  which  circumstances  within  the  system  of  systems  were  chang¬ 
ing.  In  this  case,  the  control  loop  time  constant  was  much  greater  than  the  system- 
of-systems  time  constant,  resulting  in 

•  instability  of  the  system  of  systems  (i.e.,  decisions  made  so  late  that  they  tend 
to  aggravate  the  situation,  not  mitigate  issues) 

•  schedule  delays 

•  cost  growth 

•  both  schedule  delays  and  cost  growth 

3.3.5  Recap 

As  the  previous  vignettes  and  workshop  summaries  illustrate,  different  failure 
modes  can  arise  from 

•  inappropriately  applying  program-centric  acquisition  principles  and  practices 
in  a  system-of-systems  setting 

•  lacking  a  systems  engineering  capability  at  the  system-of-systems  level 

•  centralizing  all  programmatic  decisions  among  programs  engaged  in  a  system- 

of-systems  acquisition 

•  failing  to  coordinate  acquisition  activities  with  operational  plans 

In  short,  to  be  successful  in  a  system-of-systems  context,  you  need  to  rethink  every 
aspect  of  how  decisions  are  made  throughout  the  system-of-systems  life  cycle — 
including  what  you  do,  why  you  do  it,  how  you  do  it,  and  when  you  do  it. 

The  complexity  of  the  shift  to  COTS  products,  as  challenging  as  it  was,  is  dwarfed 
by  that  brought  forth  in  the  transition  to  systems  of  systems.  Any  deficiencies  in  the 
execution  of  traditional  systems  acquisition  and  engineering  activities  are  greatly 
magnified  when  you  move  to  a  system  of  systems. 


6  In  engineering,  control  theory  deals  with  the  behavior  of  dynamical  systems.  For  some  background  on 
control  theory,  go  to  http://en.wikipedia.org/wiki/Control_theory. 
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4  Summary 


This  report  examined  interoperable  acquisition — the  set  of  practices  that  enable 
acquisition,  development,  and  operational  organizations  to  collaborate  more  effec¬ 
tively  in  fielding  interoperable  systems — from  a  variety  of  perspectives  in  the  con¬ 
text  of  real-world  acquisition  programs.  Some  of  the  key  obstacles  to  achieving 
interoperable  acquisition  were  identified,  including 

•  a  failure  to  understand  the  relationships  between  the  aspects  of  interoperability 
and  how  they — in  turn — influence  system-of-systems  acquisition 

•  a  lack  of  recognizing  that  sharing  relevant  information  in  performing  acquisi¬ 
tion-related  activities  enables  collective  behavior  that  will  successfully  deliver 
system-of-systems  interoperability 

•  the  adoption  of  system-centric  rather  than  system-of-systems  thinking  in  all 
aspects  of  acquisition,  development,  fielding,  sustainment,  and  operations — 
including  engineering  and  risk  management 

•  the  inappropriate  use  of  centralized  management,  with  hierarchical  organiza¬ 
tional  structures,  in  systems  of  systems 

•  an  absence  of  synchronization  between  the  acquisition  and  operational  com¬ 
munities  resulting  in  individual  systems — components  of  a  larger  system  of 
systems — being  developed,  acquired,  and  fielded  without  regard  for  how  they 
fit  within  the  operational  community’s  concept  of  operations 

Recognition  of  the  seemingly  inevitable  consequence  of  the  failure  to  address  in¬ 
teroperable  acquisition  necessitates  further  work  in  this  area.  The  work  to  be  done 
includes  an  examination  and  reconsideration  of  the  principles  of  acquisition,  from 
the  statutory  and  regulatory  perspective  to  the  engineering  and  use  of  systems  of 
systems.  The  current  IR&D  effort,  as  well  as  other  SEI  work,  is  addressing  these 
topics. 
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